This document is dedicated to conducting the confirmatory performance analyses that were proposed for Experiment 1.

Design and Predictions

Design. The design and analysis is a 2 (difficulty: harder than reference vs. easier than reference) X 2 (difference: moderate vs. extreme) X 2 (transition: repeat vs. switch) within-subjects ANOVA on RTs and error rates.
Predictions. There are predicted main effects of difficulty (easier deck has faster RTs and more errors than harder), and transition (typical switch cost). There is a predicted two-way interaction between difference and difficulty, such that harder decks are associated with slower RTs. There is another predicted two-way interaction between transition and difficulty, such that switch costs are expected to be greater for decks that are easier than reference. Finally, there is a predicted three-way interaction, where switch costs are expected to be larger for easier relative to harder decks. This is because switching tends to be more costly when it is done more infrequently.

Results

Response Times

Below is the cleaned data:

d <- read.csv('../../../data/dstClean.csv')
d <- d %>% 
  filter(transition != 'startBlock', chosenDeckId != 'reference') 
N <- d %>% 
  group_by(subject) %>% 
  summarize(n()) %>% 
  nrow(.)
d

The sample size is 70.

Visualize the Results

cellMeans <- d %>% 
  group_by(subject, difference, difficulty, transition) %>% 
  summarize(mrt = mean(cuedRt)) %>% 
  group_by(difference, difficulty, transition) %>% 
  summarize(rt = mean(mrt), se = sd(mrt) / sqrt(n())) 
cellMeans
cellMeans %>% 
  ggplot(aes(x = difference, y = rt, group = transition)) + 
  geom_bar(stat = 'identity', aes(fill = transition), color = 'black', position = position_dodge(width = 0.9)) + 
  geom_errorbar(aes(ymin = rt - se, ymax = rt + se), position = position_dodge(width = 0.9), width = 0.5) + 
  facet_wrap(~difficulty) + 
  theme_bw() +
  xlab('Transition Type') + 
  ylab('Response Time (ms)') + 
  scale_fill_manual(name = 'Transition', values = c(`repeat` = 'Light Grey', switch = 'Black')) +
  theme(strip.background = element_rect(color = 'black', fill = 'white'),
        legend.position = 'top')

Statistics

Subject-wise cell means

subjectCellMeans <- d %>% 
  group_by(subject, difference, difficulty, transition) %>% 
  summarize(rt = mean(cuedRt)) 
subjectCellMeans

Omnibus Model There are 15 subjects with missing cells in the design and they are being dropped from the analysis

badSubjects <- data.table(subjectCellMeans)[,.(count = .N), by = subject][count < 8, subject]
subjectCellMeans <- subjectCellMeans[!(subjectCellMeans$subject %in% badSubjects),]
m1 <- ezANOVA(wid = subject, within = .(difference, difficulty, transition), dv = rt, data = subjectCellMeans, detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: You have removed one or more levels from variable "transition".
## Refactoring for ANOVA.
cs <- c(colnames(m1$ANOVA), 'n2p')
omni <- cbind(m1, data.frame(n2p = m1$ANOVA$SSn / (m1$ANOVA$SSn + m1$ANOVA$SSd)))
colnames(omni) <- cs
omni$p <- round(omni$p, 3)
omni

Marginal Means for Main Effects

d %>% 
  group_by(subject, difficulty) %>% 
  summarize(rtime = mean(cuedRt)) %>% 
  group_by(difficulty) %>% 
  summarize(rt = mean(rtime), se = sd(rtime) / sqrt(N))
d %>% 
  group_by(subject, transition) %>% 
  summarize(rtime = mean(cuedRt)) %>% 
  group_by(transition) %>% 
  summarize(rt = mean(rtime), se = sd(rtime) / sqrt(N))

Planned Comparisons

Difficulty X Difference

## difficulty X difference
DFd <- m1$ANOVA$DFd[5]
SSd <- m1$ANOVA$SSd[5]
MSe <- SSd / DFd

omnibusParams <- c('DFd' = DFd, 'SSd' = SSd, 'MSe' = MSe)

d %>% 
  group_by(subject, difference, difficulty) %>% 
  summarize(mrt = mean(cuedRt)) %>% 
  group_by(difference, difficulty) %>% 
  summarize(rt = mean(mrt), se = sd(mrt) / sqrt(n()))
m2 <- ezANOVA(wid = subject, within = difference, dv = rt, data = subjectCellMeans[subjectCellMeans$difficulty == 'Easier than Reference',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: Collapsing data to cell means. *IF* the requested effects are a
## subset of the full design, you must use the "within_full" argument, else
## results may be inaccurate.
easier <- formatSimpleEffects(m2, omnibusParams)

m2 <- ezANOVA(wid = subject, within = difference, dv = rt, data = subjectCellMeans[subjectCellMeans$difficulty == 'Harder than reference',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.

## Warning: Collapsing data to cell means. *IF* the requested effects are a
## subset of the full design, you must use the "within_full" argument, else
## results may be inaccurate.
harder <- formatSimpleEffects(m2, omnibusParams)

data.frame(Contrast = c('Simple effect of difference for Easier than Reference', 'Simple effect of difference for Harder than Reference'),
           Result = c(easier, harder))

Transition X Difficulty

DFd <- m1$ANOVA$DFd[7]
SSd <- m1$ANOVA$SSd[7]
MSe <- SSd / DFd

omnibusParams <- c('DFd' = DFd, 'SSd' = SSd, 'MSe' = MSe)

subjectCellMeans %>% 
  group_by(subject, transition, difficulty) %>% 
  summarize(mrt = mean(rt)) %>% 
  group_by(transition, difficulty) %>% 
  summarize(rt = mean(mrt), se = sd(mrt) / sqrt(n())) 
m2 <- ezANOVA(wid = subject, within = transition, dv = rt, data = subjectCellMeans[subjectCellMeans$difficulty == 'Easier than Reference',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: You have removed one or more levels from variable "transition".
## Refactoring for ANOVA.
## Warning: Collapsing data to cell means. *IF* the requested effects are a
## subset of the full design, you must use the "within_full" argument, else
## results may be inaccurate.
easier <- formatSimpleEffects(m2, omnibusParams)

m2 <- ezANOVA(wid = subject, within = transition, dv = rt, data = subjectCellMeans[subjectCellMeans$difficulty == 'Harder than reference',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: You have removed one or more levels from variable "transition".
## Refactoring for ANOVA.
## Warning: Collapsing data to cell means. *IF* the requested effects are a
## subset of the full design, you must use the "within_full" argument, else
## results may be inaccurate.
harder <- formatSimpleEffects(m2, omnibusParams)

data.frame(Contrast = c('Simple effect of transition for Easier than Reference', 'Simple effect of transition for Harder than Reference'),
           Result = c(easier, harder))

Post-hoc follow ups on the three-way interaction

DFd <- m1$ANOVA$DFd[8]
SSd <- m1$ANOVA$SSd[8]
MSe <- SSd / DFd
omnibusParams <- c('DFd' = DFd, 'SSd' = SSd, 'MSe' = MSe)

m2 <- ezANOVA(wid = subject, within = .(transition, difference), dv = rt, data = subjectCellMeans[subjectCellMeans$difficulty == 'Easier than Reference',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: You have removed one or more levels from variable "transition".
## Refactoring for ANOVA.
easier <- formatSimpleEffects(m2, omnibusParams)

m2 <- ezANOVA(wid = subject, within = .(transition, difference), dv = rt, data = subjectCellMeans[subjectCellMeans$difficulty == 'Harder than reference',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.

## Warning: You have removed one or more levels from variable "transition".
## Refactoring for ANOVA.
harder <- formatSimpleEffects(m2, omnibusParams)

data.frame(Contrast = c('Simple effect of transition X difference for Easier than Reference', 'Simple effect of transition X difference for Harder than Reference'),
           Result = c(easier, harder))

Testing for simple effect of difference across levels of transition (for easier than reference only)

SSd <- m1$ANOVA$SSd[2]
DFd <- m1$ANOVA$DFd[2]
MSe <- SSd / DFd
omnibusParams <- c('DFd' = DFd, 'SSd' = SSd, 'MSe' = MSe)

m2 <- ezANOVA(wid = subject, within = difference, dv = rt, data = subjectCellMeans[subjectCellMeans$difficulty == 'Easier than Reference' & subjectCellMeans$transition == 'repeat',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
repetition <- formatSimpleEffects(m2, omnibusParams)

m2 <- ezANOVA(wid = subject, within = difference, dv = rt, data = subjectCellMeans[subjectCellMeans$difficulty == 'Easier than Reference' & subjectCellMeans$transition == 'switch',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
Switch <- formatSimpleEffects(m2, omnibusParams)

data.frame(Contrast = c('Simple effect of difference for repetitions and Easier than Reference', 'Simple effect of difference for switches and Easier than Reference'),
           Result = c(repetition, Switch))

Error Rates

Below is the cleaned data:

d <- read.csv('../../../data/dstCleanErrors.csv')
d <- d %>% 
  filter(transition != 'startBlock', chosenDeckId != 'reference')
d

Visualize the Results

cellMeans <- d %>% 
  filter(transition != 'startBlock') %>% 
  group_by(subject, difference, difficulty, transition) %>% 
  summarize(merror = mean(error)) %>% 
  group_by(difference, difficulty, transition) %>% 
  summarize(error = mean(merror), se = sd(merror) / sqrt(n())) 
cellMeans
cellMeans %>% 
  ggplot(aes(x = difference, y = error, group = transition)) + 
  geom_bar(stat = 'identity', aes(fill = transition), color = 'black', position = position_dodge(width = 0.9)) + 
  geom_errorbar(aes(ymin = error - se, ymax = error + se), position = position_dodge(width = 0.9), width = 0.5) + 
  facet_wrap(~difficulty) + 
  theme_bw() +
  xlab('Transition Type') + 
  ylab('Error Rate') + 
  scale_fill_manual(name = 'Transition', values = c(`repeat` = 'Light Grey', switch = 'Black')) +
  theme(strip.background = element_rect(color = 'black', fill = 'white'),
        legend.position = 'top')

Statistics

Subject-wise cell means

subjectCellMeans <- d %>% 
  group_by(subject, difference, difficulty, transition) %>% 
  summarize(error = mean(error)) 
subjectCellMeans

Omnibus Model

badSubjects <- data.table(subjectCellMeans)[, .(count = .N), by = subject][count < 8, subject]
subjectCellMeans <- subjectCellMeans[!(subjectCellMeans$subject %in% badSubjects),]
m1 <- ezANOVA(wid = subject, within = .(difference, difficulty, transition), dv = error, data = subjectCellMeans, detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: You have removed one or more levels from variable "transition".
## Refactoring for ANOVA.
cs <- c(colnames(m1$ANOVA), 'n2p')
omni <- cbind(m1, data.frame(n2p = m1$ANOVA$SSn / (m1$ANOVA$SSn + m1$ANOVA$SSd)))
colnames(omni) <- cs
omni

Marginal Means for Main Effects

print('Main effect of switch cost')
## [1] "Main effect of switch cost"
d %>% 
  group_by(transition, subject) %>% 
  summarize(err = mean(error)) %>% 
  group_by(transition) %>% 
  summarize(error = mean(err), se = sd(err) / sqrt(N))
print('Overall error rate')
## [1] "Overall error rate"
d %>% 
  group_by(subject) %>% 
  summarize(err = mean(error)) %>% 
  summarize(error = mean(err), se = sd(err) / sqrt(N))

Planned Comparisons

Difficulty X Difference

## difficulty X difference
DFd <- m1$ANOVA$DFd[5]
SSd <- m1$ANOVA$SSd[5]
MSe <- SSd / DFd

omnibusParams <- c('DFd' = DFd, 'SSd' = SSd, 'MSe' = MSe)

d %>% 
  group_by(subject, difference, difficulty) %>% 
  summarize(mrt = mean(cuedRt)) %>% 
  group_by(difference, difficulty) %>% 
  summarize(rt = mean(mrt), se = sd(mrt) / sqrt(n())) 
m2 <- ezANOVA(wid = subject, within = difference, dv = error, data = subjectCellMeans[subjectCellMeans$difficulty == 'Easier than Reference',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: Collapsing data to cell means. *IF* the requested effects are a
## subset of the full design, you must use the "within_full" argument, else
## results may be inaccurate.
easier <- formatSimpleEffects(m2, omnibusParams)

m2 <- ezANOVA(wid = subject, within = difference, dv = error, data = subjectCellMeans[subjectCellMeans$difficulty == 'Harder than reference',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.

## Warning: Collapsing data to cell means. *IF* the requested effects are a
## subset of the full design, you must use the "within_full" argument, else
## results may be inaccurate.
harder <- formatSimpleEffects(m2, omnibusParams)

data.frame(Contrast = c('Simple effect of difference for Easier than Reference', 'Simple effect of difference for Harder than Reference'),
           Result = c(easier, harder))

Transition X Difficulty

DFd <- m1$ANOVA$DFd[7]
SSd <- m1$ANOVA$SSd[7]
MSe <- SSd / DFd

d %>% 
  group_by(subject, transition, difficulty) %>% 
  summarize(mrt = mean(cuedRt)) %>% 
  group_by(transition, difficulty) %>% 
  summarize(rt = mean(mrt), se = sd(mrt) / sqrt(n())) 
m2 <- ezANOVA(wid = subject, within = transition, dv = error, data = subjectCellMeans[subjectCellMeans$difficulty == 'Easier than Reference',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: You have removed one or more levels from variable "transition".
## Refactoring for ANOVA.
## Warning: Collapsing data to cell means. *IF* the requested effects are a
## subset of the full design, you must use the "within_full" argument, else
## results may be inaccurate.
easier <- formatSimpleEffects(m2, omnibusParams)

m2 <- ezANOVA(wid = subject, within = transition, dv = error, data = subjectCellMeans[subjectCellMeans$difficulty == 'Harder than reference',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: You have removed one or more levels from variable "transition".
## Refactoring for ANOVA.
## Warning: Collapsing data to cell means. *IF* the requested effects are a
## subset of the full design, you must use the "within_full" argument, else
## results may be inaccurate.
harder <- formatSimpleEffects(m2, omnibusParams)

data.frame(Contrast = c('Simple effect of transition for Easier than Reference', 'Simple effect of transition for Harder than Reference'),
           Result = c(easier, harder))

I’ll deal with the three-way only if I need to.

 

Analysis Homepage

A work by Dave Braun

dab414@lehigh.edu